Vision Algorithms Catch Defects in Screen Displays 


NASA Technology 

NASA has sent more than a few robotic missions into space, but it never loses sight of its goal to enable 
human exploration of the cosmos. A core component of planning for future manned missions is the 
Human Systems Integration Division, headquartered at Ames Research Center, which focuses on 
advancing our understanding of how people process information and interact with mechanical and 
electronic systems. 

Andrew Watson is Ames’ Senior Scientist for Vision Research and the director of the Vision Group at the 
Human Systems Integration Division. “Many NASA missions in both space and aeronautics rely on 
vision,” he says, “either to monitor the environment around us or to gather information from visual 
displays. It is important to understand how the limits and capabilities of vision constrain and enable 
human performance in these missions.” 

Part of Watson’s research focuses on optimizing vision technology, including the design of robotic vision 
systems. One of the long-standing problems in robotic vision is predicting the visibility of arbitrary 
targets, so Watson coordinated an international effort involving 10 research labs to collect data on it and 
developed a computer model that could predict those data. 

“I also knew of many applied problems that would benefit from a simple model of target visibility,” he 
says, “so to serve that need, I converted the science model into an applied tool.” 

Watson named his tool the Spatial Standard Observer (SSO), which has since been patented. The SSO is a 
simplified model of human vision, focusing on how contrast is perceived by the human eye. According to 
Watson, the SSO has potential applications in many areas, including vision for unmanned aerial vehicles, 
predicting outcomes for laser eye surgery, evaluating the legibility of text, and more. 

Technology Transfer 

One of the SSO’s applications soon caught the eye of Redmond, Washington-based Radiant Zemax LLC, 
a company that provides tools for designing and testing various optical and display systems. At the time, 
the company was developing software tools to support its imaging colorimeter, a camera that measures 
color and brightness in accordance with the International Commission on Illumination’s standards for 
human visual performance. 

A typical high-defmition display, like that on a flat-panel television, contains approximately 2 million 
pixels, any one of which could be defective — what are termed “dead” or “stuck-on” pixels. Dead pixels 
and defects like them are relatively easy to catch when testing manufactured units, whether the inspector 
is a camera, sensor, or human being. But there is another class of defect called blemishes, which are areas 
of changing brightness or color that could be anywhere on the display and rather subtle. (The display 


industry has largely adopted the Japanese word mura — which means blemish — to describe these.) As 
mura have no necessary, fixed qualities, they tend to be difficult to predict or find. 

Says Hubert Kostal, vice president of sales and marketing at Radiant Zemax, “As we researched this 
problem, we asked, ‘How can we find these floating defects, which can be anywhere in the display image 
and be any shape?’ If you know what you’re looking for, you can do a scan for it, but you get to a point 
where you’re not even sure whether it will be oval-shaped, square-shaped, or a bunch of dot defects 
clustered together.” 

What Radiant Zemax needed was an objective way to classify the shape of luminance and color 
differences as human beings see them, in a way that could be incorporated into its image analysis 
software. The industry-standard method of grading differences in color is the use of a just noticeable 
difference (JND) scale, with one unit of JND representing the minimal change in color between two areas 
that is required before an average human observer could notice the difference between them. And, says 
Kostal, “as it turned out, one of the outputs from Watson’s approach allowed us to use JND to assign 
numerical values to the severity of blemishes.” 

Satisfied that it had found the right tool for the job, Radiant Zemax licensed the technology from NASA, 
combined some of the SSO algorithms with the company’s own software, and created a new product 
called TrueMURA. 

Benefits 

Manufacturers of television, laptop, tablet, and phone displays usually operate on razor-thin profit 
margins and face a dilemma when inspecting their product: On the one hand, an inspection process that 
doesn’t detect flawed units will result in returns, repairs, and unhappy customers unlikely to buy from the 
company again. On the other hand, imaging colorimeters can be more sensitive to flaws than the human 
eye, and manufacturers lose a lot of money when they throw away good product that could have satisfied 
consumers — even if the inspection flagged it as potentially defective. 

“That’s the conundrum,” says Kostal. “When is a mura meaningful to a human observer? How severe is 
it?” With its NASA-improved software, Radiant Zemax helps display manufacturers ensure that they are 
catching enough of the right kind of mura. “The most important thing is not what defects you can get a 
camera to detect,” says Kostal. “What’s really important is what customers see; it’s only a defect when a 
customer can see it.” 

TrueMURA works with Radiant Zemax’ s existing imaging colorimeters and software tools and, 
according to the company, is the first commercial system to provide image analysis algorithms for 
computing a value for JND that matches human visual perception. It ignores mura that cannot be seen by 
humans and categorizes other blemishes according to how noticeable they are to human observers. 

TrueMURA can take over functions that have traditionally required human inspection, which is known to 
produce imprecise judgments. “All kinds of variables come into play with human inspection,” says 
Kostal. “If I’m giving you 1,000 displays to inspect, you might only have 5-10 seconds to look at each 
one. A defect that barely reaches the level of human perception might not be seen by an inspector, but a 


consumer watching hours of television and movies on that display is definitely going to notice it.” Kostal 
also points out that, when compared to the cost of using human inspectors, companies using Radiant 
Zemax’s product will end up saving money within 1 year. 

The SSO-based TrueMURA software is currently used by a large number of companies producing 
common consumer electronics like smartphones, tablets, laptops, and televisions. “I can guarantee you,” 
says Kostal, “that if you go into a major electronics store, you will find products there that were either 
tested or designed using this capability.” 

The company has been very pleased with its experience licensing NASA technology. “We do really cool 
things at Radiant Zemax, but we’re fundamentally an engineering company,” says Kostal. “We want to 
develop and provide solutions to our customers. The information that we were able to get from NASA 
enabled us to move quickly to the forefront of the industry in terms of analysis, methodology, and 
techniques.” 


TrueMURA™ is a trademark of Radiant Zemax LLC. 


